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PREFACE 



The National Science Foundation supported more than 600 inservice teacher training 
projects between 1984 and 1989 under its Teacher Enhancement Program (TEP) 1 . This is but 
one component of a comprehensive NSF effort to assure that the nation' s future supply of qualified 
scientists, mathematicians, and engineers is adequate for the challenges ahead and that American 
citizens are able to function effectively in a technologically intensive world. 

In 1990, the Division for Research, Evaluation and Dissemination (RED) 2 undertook 
the first of two studies of TEP, in which a survey was conducted of the 600 Principal Investigators 
(Pis) who had operated inservice teacher enhancement projects. This survey sought to learn about 
their goals for their TEP projects, who they sought to reach, how participants were recruited and 
from what types of schools, how Pis ran their projects, who helped them, and how Pis assessed 
the impact of their projects upon the participants. Subsequently, a second survey was conducted 
in 1992 of individuals who had participated in these projects in order to learn, first hand, from 
the participants, about their experiences with the TEP projects and what they believed were the 
major impacts. The findings from both of these studies have been merged in the present report. 

Several lessons have been learned in the course of conducting these studies and are worth 
notingattheoutset. First, these two studies of TEP projects provide an unusual program evaluation 
opportunity in that this report builds upon data and observations provided both by the TEP project 
Pis and by participants in the TEP projects. The second study of participants obtained information 
complementary to the earlier survey of Pis. The perspectives of participants on many of the same 
questions asked of Pis provide a form of validation regarding earlier Pis' assessments of project 
impacts. If both Pis and participants independently confirm that something was accomplished 
by a project, we may have greater confidence in the ability of Pis to assess the impacts of their 



l ln this report, the acronym TEP is used to designate the Teacher Enhancement Program. Within the National 
Science Foundation, the acronym TE is used. 

2 At that time, the name for this division was the Office of Studies, Evaluation and Dissemination (OSED). 



projects on their participants. In fact, many such instances of correspondence were found between 
responses provided by these two study groups (and are reported upon throughout this volume). 

Second, in this report it is recognized that participants, most of whom are practicing 
teachers, do not necessarily constitutea homogeneous population. Teachers in elementary schools 
are unlikely to have the same needs with respect to inservice learning as their counterparts from 
secondary schools. In order to deal with this consideration, various relationships among the findings 
are examined and differences among participants as a function of grade level are noted. 

Third, since the awards examined cover a six year time frame, the projects themselves 
may have changed over time. Thus, data were analyzed by cohort year and where changes occurred 
over time, they are commented upon. 

This volume provides the reader a summary of many of the key findings from this 
evaluation. It was designed to be reviewed quickly in order to find out what was learned from 
the two studies. However, much of the richer detail had to be omitted. The companion Volume 
n provides more technical detail on the studies. Volume II is designed for use by two audiences. 
First, for NSF program officers , Volume n provides extensive details on the study design, findings 
on many separate topics, the relationships among various findings, and conclusions and 
recommendations. In addition, special exhibits are included that provide quotations from both 
Pis and participants on a variety of questions concerning the impacts of the TEP projects, lessons 
learned, "best" models for in-service programs, and so forth. 

Second, for the technically oriented reader, Volume n provides details on such matters 
as sampling and weighting procedures, and complete statistical tables and copies of the questionnaire 
are provided. 

Finally, both databases are available in electronic form to permit additional analyses 
of specific questions that educators and scientists may have. 

Recommendations derived from the surveys have been made with two objectives in mind. 
First, to suggest any improvements in the operation of the program itself that will, over the long 
run, further enhance its effectiveness. Second, to ensure that subsequent evaluation efforts will 
contribute the most policy- and program-relevant information to the staff of NSF. It is important 
to note that the recommendations presented in this report are based upon the responses to various 
questions posed to Pis and participants involved with TEP projects during the period between 
1984 and 1989. In turn, these awards were operating under program guidelines issued for applications 
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made during those fiscal years. Thus, these recommendations do not take into account changes 
in the TEP policies, program, and guidelines made in subsequent years. In the interim period, 
a number of changes, in fact, have been made. According to NSF officials, "since the time of 
this study, the program has nearly doubled in length of training and the teacher stipend; allows 
for 5-year projects; requires more evaluation and follow-up; requires administrative support in 
home institutions; and focuses more on leader teachers and district-wide system projects." In 
this sense, a number of recommendations in this report provide a form of external support to changes 
already made. 

Readers wishing to obtain more details on this study are invited to contact the Division 
of Research, Evaluation and Dissemination, Directorate for Human Resources, National Science 
Foundation, in Washington, D.C. 



ERIC 



7 

V 



SUMMARY OF FINDINGS 



Enhancing Science and Mathematics Education in America 

In response to a growing national consensus on the need to improve the quality of 
science and mathematics education provided to our nation's school children, the National Science 
Foundation (NSF) initiated its Teacher Enhancement Program (TEP) in 1984. The goal of 
TEP is to enhance the quality of education provided to the nation's school children by supporting 
effective in-service education programs for science and mathematics teachers in elementary, middle, 
and senior high schools throughout the country. 

Through TEP, NSF is also interested in promoting participation among groups presently 
underrepresented in the sciences, including women, minorities, and persons with physical disabilities. 
A further goal of the TEP is to foster the development and dissemination of improved models 
for in-service education programs for science and mathematics teachers across the country. Finally, 
NSF seeks to increase the impact of its programs by encouraging financial and in-kind support 
from other sources. 

During the first six years of the program, 599 TEP projects were funded throughout 
the United States, withatotal Federal commitment of $160 million. In 1991 and 1992, NSFcarried 
out an evaluation of this program both to assess its accomplishments and to determine appropriate 
future policies, program operations procedures, and funding levels. A summary overview of the 
analytic logic that guided the evaluations appears in Figure 1. Based upon a survey conducted 
of all Principal Investigators (Pis) who received gnuits between Fiscal Years 1984 and 1989, and 
a second survey conducted of a sample of TEP participants, weconclude that many of the objectives 
established for this program are being achieved 3 . 



'Seventy-six percent of the Pis, and 59 percent of TEP participants responded to the respective surveys. The 
information obtained was used in preparing estimates for the entire population, which are reported in this summary. 
Estimates for the total population were produced based upon several known population characteristics (including 
award year, number of participants, grade levels taught by participants, and expiration date of award). 




Key findings 

Based upon records the Pis had regarding actual participation, it is estimated that NSF 
has supported the training of more than 63,000 science and mathematics teachers under awards 
made between FY1984 and 1989. This is the equivalent of 3.5 % of all pre-college science and 
mathematics teachers in the United States 1 . In their assessment of the TEP experience, the great 
majority of TEP participants (81 %) agreed that they would strongly recommend participation 
in a TEP project to another teacher. Many participants expressed deep gratitude for the inservice 
opportunity afforded them by TEP; one teacher indicated that participating in the TEP project 
"was the single most powerful, professional thing I have done. It changed the way I teach everything. 
Thank you!" 2 

A typical TEP project operated for between two and three years and provided training 
to just over 100 teachers. Among the 599 TEP projects that provided in-service education, over 
90% operated during the summer. According to Pis, participants typically received 120 hours 
of in-service training in a summer, and many participants returned for a second summer. More 
than 85 % of the projects also offered a program during the school year, in which typically 24 
hours of in-service training were provided. Finally, post-summer independent study was reported 
by a quarter of the Pis (an average of 25 hours). 

In-service education focused principally on biological, physical, and earth sciences 
and mathematics, although other sciences were covered to a lesser degree. The specific emphasis 
varied somewhat as a function of grade level (see Figures 2 and 3). 

The typical TEP project director had earned a doctorate, was educated in a field of 
science, mathematics, or science education (see Figure 4), taught at apublic orprivate university, 
and had tenure. Project leaders (and TEP staff) were drawn principally from institutions of higher 
education, although pre-college teachers and staff from museums and other local institutions also 
provided in-service training. 
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1 It is estimated by the Division of Teacher Preparation and Enhancement of NSF that there are approximately 
2,000,000 pre^coliege science and mathematics teachers in the United States. However, because some teachers may 
have participated in more than one TEP project, it is not possible to determine the exact percentage of the population 
of teachers that participated in TEP. 

2 Many comments on the TEP projects were provided by both Pis and participants regarding various facets of 
the TEP project and their experiences. These may be found in Volume II, Chapter II of this report. 
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Figure 2: Areas of Science Receiving Major Emphasis in 
TEP Projects, by Grade Level 
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Figure 3: Areas of Mathematics Receiving Major Emphasis in 
TEP Projects, by Grade Level 
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Figure 4 - Discipline of Awardees' Highest Degree 
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Over the years, the goals most frequently cited by TEP Ms for their projects include: 
improving teachers' knowledge of science and mathematics content; providing teachers with experience 
in hands-on instructional activities; and developing ways for teachers to enhance student interest 
in science or mathematics (see Figure SA). In recent years, the development of teacher skills for 
improving student problem-solving, and attention to enhancing student interest in science and 
mathematics have emerged as central goals. The Pis also reported having a variety of more general 
and organizational goals, including the addressing of local and regional needs, the evaluation of 
their projects' effects, insuring the continuance of support of projects upon completion of the grant, 
and establishing local partnerships among science and mathematics organizations. 

Participants were also asked about their own goals and objectives for their projects 
(see Figure 5B). The participants and Pis stressed the importance of many of the same goals. 
The objectives cited most often by participants include developing ways to enhance student interest 
in mathematics and science, increasing their own knowledge of how to apply math and science 
principles, gaining experience with hands-on activities, and improving their knowledge of math 
and science content . Participant rankings of their objectives, over time, did not fluctuate greatly. 

In terms of instructional approaches used by Pis, as shown in Figure 6A, 86% emphasized 
hands-on activities, while 45% emphasized small group discussions, and 43% the development 
of student instructional materials. The TEP participants were also asked about the instruction 
methods and materials used in the TEP project. As seen in Figure 6B, participants confirmed 
that hands-on activities are the most often used method in TEP projects; 77% of participants reported 
that hands-on activities were used often in their projects. Other teaching methods used, according 
to participants, included cooperative learning groups (42%), lectures (42%), instructor demonstrations 
(39%), and small group discussion sections (35%). 

Pis were asked to assess the effectiveness of various in-service education methods. 
In the eyes of the Pis, effective in-service training projects emphasize hands-on activities, field 
trips, and the use of cooperative learning groups (see Figure 7A). Development of student instructional 
materials, small group discussions, use of resource people, instructor demonstrations, and peer 
teaching also were seen as effective instructional methods. Role playing, lectures, computer assisted 
instruction, library research, films and videos were seen as less valuable instruction devices. An 
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Figure 5A - Most Frequently Held Goals and Objective! 
(Percentage of Pis Rating Goal as 1st, 2nd, or 3rd) 
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Figure SB - Participants' Most Frequently Held Goals and Objectives 
(Percentage of Pis Rating Goal as 1st, 2nd, or 3rd) 
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Figure 6A - Pi-Reported Instructional Methods and Materials 

Used in TEP Project 
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Figure 6B * Participant-Reported Instructional Methods and Materials 

Used in TEP Project 
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Figure 7A - PI Assessments of the Usefulness of Various 
Teaching Methods and Materials (Percentage Reporting as " Very Effective") 
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important exception, however, was that lectures were seen as the most effective method for improving 
content knowledge levels. The assessments provided by the participants themselves (Figure 7B) 
show general correspondence to the Pi's effectiveness assessments. 

The Pis reported that their major project accomplishments included increasing participants' 
knowledge of science and mathematics, the subsequent use of new materials by participants in 
the classroom, and increased knowledge among participants of the applications of science and 
mathematics (see Figure 8A). 

Participants were af Jced to rate their TEP projects on a broad range of criteria (see 
Figure 8B). Four in five would recommend the program to other teachers. Participant rating 
was similar to those of the Pis. 

A critical concern is the extent to which changes occurred in teaching methods and 
contents among TEP participants upon return to the classroom. As shown in Figure 9, 80% of 
participants reported that, since TEP, they have provided students with more hands-on laboratory 
experiences, 75% had them work more in small groups, 71% integrated the applications of 
mathematics and science into their teaching more, and 69% helped students more to find answers 
to their own questions. Two other changes are also of note. Sixty-one percent conducted more 
scientific demonstrations for their students, and 52% use computers more in the classroom. In 
contrast, 35 % of participants reported that they have decreased the amount of time spent lecturing 
to a whole class, and decreased the time that students spent reading textbooks in class. On average, 
about one-third of participants cited the TEP experience as an important contributing factor to 
the change. 

Participants were asked whether they believe the changes in their teaching have, in 
turn, been reflected in changes among their students. According to TEP participants' perceptions 
of such changes (see Figure 10), they have noticed increased student enthusiasm in class (80%), 
an increase in classroom test scores (52%), increased interest among students in careers in 
mathematics and science (49%), increased involvement in projects outside of class (43%), and 
increased participation in science fairs or contests (35%). Generally speaking, between one-half 
and two-thirds of the participants believed that their own TEP experience had had a major or 
relatively great impact upon the specific changes in student behaviors that they had observed. 
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figure 7B - Participant Assessments of the Usefulness of Various 
Teaching Methods and Materials (Percentage Rating as "Very Effective 91 ) 
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Figure 8A * Principal Investigators' Assessments of Project Outcomes 
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Figure 8B - Participants' Assessments of Project Outcomes 
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Figure 9 - Participants' Reports of Changes in Their Teaching Activities 
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Figure 10 - Changes in Student Characteristics Since TEP Experience 
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Participants were also asked to indicate whether or not they had established working 
partnerships with other institutions. Across all years, 40% of participants reported that they had 
formed a working partnership with a scientific institutional resource. Among those reporting 
such partnerships, the most frequently identified types of institutional connections were with university 
science faculty (25%), university education faculty (17%), university faculty in mathematics or 
computer science (11%), and persons from private industry (11%). 

Participants reported on a variety of post-project professional activities (see Figure 
11). The most common post-TEP post-project activities cited by participants were serving as 
a mentor to other teachers within their school (60%), making presentations to other teachers outside 
of their school (44%), and contacting someone at a college or university (34%). Many former 
TEP participants have been promoted to leadership positions: 47% of participants have been 
named mentor/master teachers, 41 % have been named subject area department heads, and 25% 
have begun serving as science or mathematics curricula specialists since participating in TEP. 
Former TEP participants have also moved into leading administrative roles: 3.5 % of TEP participants 
reported that they have been promoted to principal, and 8 % reported that they are now performing 
other administrative roles in their school districts. 

NSF strives to reach members of undenepresented groups through its many educational 
programs. Participants were asked to indicate their own race or ethnicity. Blacks (not of Hispanic 
origin) constituted 7.9% of all participants, while3.3% were Hispanic, 2.2% were Asian or Pacific 
Islander, and 1 . 1 % were American Indian or Alaskan Native. All of these percentages are slightly 
lower than the corresponding estimates provided by the Pis, who reported that an aggregate of 
17.6% of all participants were minorities. More than half of all TEP participants were female. 
Roughly 3% of participants reported having some form of physical disability. 

Financial Support 

Over the entire sue years studied here, the median value of an NSF award for a TEP 
project was $187,000. Significant growth occurred in award size over time, however, in 1984, 
the median award value was $77,000; this figure grew more than 300% to $309,000 by 1989. 
During the same period, the duration of a typical award increased from 24 to more than 30 months. 
Thus, the median annual value of the awards increased from $39,000 in 1984 to $124,000 by 
1989 (by more than 200%). 
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Figure 11 - Psrtidpmnts' Reports of Post-Project Activities 



Served m mentor to other 
teachers within my school 

Made presentations to other 
teachers outside of my school 

Contacted someone at a 
college or university 

Made a presentation at a 
professional meeting 

Serbed ss mentor to other 
teachers outside of my school 

Appointed as teem leader, 
chair, or mentor 

Contacted mem be r s of local 
business community 

Ogranized in-service activity 
outside of my school 

Organized in-service activity 
within my school 

Served as a staff member in a 
summer program 

Wrote for professional 
publications 

Used a computer network to 
maintain contacts 



Other similar type of activity 




10 



20 



30 



40 



50 



60 



70 



Source: Participant Survey - Item 1 1 



20 



28 



The Pis were able to achieve significant leverage on NSF funds; financial contributions 
from other sources amounted to $38.9 million (equal to 29% of NSF funds). "In-kind" contributions 
were reported with an imputed value of $32.9 million (equal to 23 % of NSF funds). This constitutes 
a combined leverage among TEP projects of 52% of NSF funding. NSF's efforts under TEP 
to improve the quality of education are being significantly enhanced through this additional support 
provided by other science and educational institutions. 

Taking into account this additional support, total annual expenditures almost doubled 
over the six year period ~ from $82 thousand in 1984 to $162 thousand in 1989. It is important 
to note that the value of both direct financial support and in-kind contributions received by TEP 
projects has risen over time. 

As shown in Figures 12 and 13, this additional support typically was provided by the 
host institutions of the TEP Pis, the school systems of the participants, foundations, private business 
and industry, various local institutions (e.g., museums), and other governmental organizations 
(including Federal R&D laboratories). 

Characteristics of the TEP Participants 

The Pis typically recruited teachers that they expected to become school leaders, of 
whom a roughly equal number were seen as being either under-prepared or well-prepared for 
their present teaching responsibilities. Additionally, female and minority participants, and participants 
from economically disadvantaged districts were sought (see Figure 14). Typically, Pis recruited 
teachers through announcements sent to local school districts, making contacts at local meetings 
of school professional staff, and sending information to local teacher organizations (see Figure 
15). Over time, the emphasis placed on recruitment of women, minorities, and teachers working 
in economically disadvantaged areas increased. Participants reported on how they first learned 
about the TEP projects in which they participated. The most common sources of information 
about TEP for participants were their local school districts (49%), other teachers or principals 
(40%), and professional journals or newsletters (21 %). Over time, the importance of local school 
districts as sources of information about TEP for prospective participants generally increased. 
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Figure 12 - Percentage of Pis Reporting Other Sources of Financial 
Support to NSF Sponsored TEP Projects, by Source 
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Figure 13 - Percentage of Pb Reporting Other Sources of In-Kind 
Support to NSF Sponsored TEP Projects, by Source 
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Figure 14 - Focus of Teacher Recruitment 
(Percentage of Pis Rating Group as a Mitfcr Focus) 
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Figure 15 - Methods Used for General Recruitment 
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Pis had provided estimates of the distribution of their project participants by grade 
level taught (see Figure 16). Participants also provided data on the level of school they taught 
(see Figure 17). The majority of participants (56%) reported teaching on the high school level, 
while just under 20% reported teaching respectively at the middle/junior high school level and 
at the elementary level. The remainder taught in Kindergarten. The figures provided by Pis varied 
somewhat from those provided by the participants (indicating somewhat lower high school and 
somewhat higher junior high school levels among their participants). 

The great majority of all TEP participants (92 %) are teachers in public school districts; 
the remaining 8 % teach in private schools. Over time, the percentage of TEP participants from 
private schools has increased steadily from 5.5% in 1984 to 11.3% in 1990. 

Participants were asked to indicate all degrees that they held; 53 % of all participants 
hold a Bachelor of Arts/Sciences, 46% have a Masters Degree in Education, 27 % have a Masters 
of Arts/Sciences, and 26% hold Bachelor degrees in Education. Over time, the distribution of 
degrees held by participants remained generally even although the percentage of participants with 
Bachelors in Arts/Sciences did increase from 49% in 1984 to 57% in 1990. In the opinion of 
Pis, the level of formal training in science or mathematics among the TEP participants varied 
directly as a function of grade level taught, with high school teachers possessing the most extensive 
undergraduate preparation in such fields. 

Participants also specified, for the highest degree earned, both their major and minor 
fields of study. The most often cited major fields of study for participants were education (31%), 
the biological sciences (18%), science education (11 %), and mathematics education (10%). The 
most common minor fields of study cited by TEP participants were education (19%), science 
education (14%), and chemistry (12%). Over time, the percentage of participants majoring in 
biological science dropped slightly while the percentage of participants earning their highest degrees 
in education increased. 

The majority of participants (57%) considered themselves to be well-prepared in-service 
teachers, while 38% said that they expected to assume leadership roles in their school districts, 
and 20% described themselves as under-prepared in-service teachers. Over time, however, more 
participants characterized themselves as under-prepared in-service teachers and fewer participants 
characterized themselves as well-prepared- 
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Figure 16- PI Report on Average Grade Level Taught by TCP Project 

Participants 




Figure 17 - Participants Report on Average Grade Level Taught by TEP 
Project Participants, by Award Cohort 
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Relationships Among Project Characteristics and Outcomes 

Various PI, TEP project, and participant characteristics were examined in order to 
determine whether significant associations existed. A variety of significant relationships were 
found. In this section, we examine first selected relationships identified among participants, and 
then turn to an examination of relationships among the Pis. 1 

Participants 

TEP participants shared several major goals when they decided to join the project. 
Sixty percent or more of participants rated each of the following goals as "very important" (the 
highest scale rating available to them): 

• Develop ways to enhance student interest 

• Gain experience with hands-on activities 

• Increase knowledge of applications of principles 

• Improve knowledge of science/math content 

• Develop additional teaching skills 

Other goals, however, were less widely shared among participants. Overall, only 
27% of teachers thought that learning to motivate female, minority, or disabled students was very 
important. Women gave this goal substantially higher importance than did men; one-third of 
the women thought it was very important, compared with one-fifth of the men. 

Among teachers who were themselves members of a minority group, however, 58% 
rated this as a very important goal influencing them to participate in TEP, compared with 23% 
of non-minority teachers. 

Both black and white teachers expressed support for this goal in direct relation to the 
number of minority children in the student bodies of the schools where they taught. More than 
half of the teachers in schools with predominately minority enrolment thought motivating these 
students was very important, while fewer than a quarter of those in schools with predominately 
non-minority enrolment thought so. Two other school attributes were also highly correlated with 



Relationships reported in Ms section were significant at the . 001 level; see Volume il of this report for a more 
detailed discussion cf these findings. 
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response to this goal: urban school locations, and fraction of students eligible for free or subsidized 
lunches. 

Teachers in poor, urban, or predominately minority schools, and teachers who were 
themselves members of minority groups, were much more strongly motivated by the prospect 
of developing the ability to mobilize other resources than were their wealthier counterparts. In 
schools where nearly all (over 75 %) of the students met the eligibility criteria for free or reduced 
price lunches, 27% of the teachers thought that mobilizing resources was an important goal, while 
in schools with few such students, and in predominately white schools, only 10-13% thought so. 
Minority teachers were substantially more influenced by this goal (30% thought it very important) 
than white teachers (14%). 

Teachers at different grade levels had substantially different motivational patterns for 
participation. Nearly half of the high school teachers thought that " [increasing] knowledge of 
cutting edge research" was a very important goal, while only a quarter of teachers of grades K-4 
rated it this high. In contrast, 43% of the K-4 teachers thought that learning how to integrate 
science or math with other subjects was very important, while at the high school level only 16% 
of teachers thought so. Elementary teachers also attached higher importance to "gainftng] experience 
with hands-on activities" and M develop[ing] ways to enhance student interest" than did high school 
teachers. 

Teachers in rural schools differed from those in cities and suburbs in their goals for 
enhancing their personal credentials. Although no group gave very high ratings to earning continuing 
education or graduate degree credits, rural teachers 9 ratings of this issue were significantly higher 
than those of teachers in urban ox suburban settings. 

Teachers were asked to rate the project they attended on 8 specific and 1 general quality 
indicators. Most teachers rated their projects as excellent on most indicators, so these ratings 
did not reveal many detailed differences among teachers or project types. In addition, ratings 
on each specific rating item were highly correlated with the respondent's general rating of the 
overall quality of the program. Among the individual items, responsiveness of staff to participants 
stands out as most highly correlated with overall satisfaction. 
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Teacher ratings were influenced by the subject matter coveted by the project, the grade 
levels targeted by project administrators, and the teachers 1 stated goals in attending the project. 
Other factors, such as the size and ethnic composition of the participating teacher population, 
and the scope of project recruitment, were not generally related to expressed teacher satisfaction 
on this item. 

On most of the scales, projects which focused on either math or science instruction 
were rated higher than those which targeted both, or one where the subject matter could not be 
determined. Among projects which specifically targeted either math or science, those which also 
specifically targeted a narrow range of grade levels tended to receive higher satisfaction ratings 
than those which recruited teachers from all grade levels (or whose grade levels could not be 
determined). 

This was especially true for lower elementary grade math teachers. Teachers of 
kindergarten through fourth grade students who attended math programs aimed specifically at 
their grade level were nearly unanimous in assessing the overall quality of their projects as excellent. 
Only 70% of similar teachers in projects with no discernible grade level focus gave this rating. 
Lower grade teachers in math projects with no specific grade focus gave particularly low ratings 
to "Knowledge of staff members about schools and teaching. " Fewer than half of the lower grade 
participants in such projects thought their staff were excellent on this measure, compared with 
90% of the lower grade teachers in grade-specific math projects. 

A related pattern emerges for high school teachers. Those teachers attending TEP 
projects specifically focused on either math or science were not particularly sensitive to the grade 
level range of participants, often rating mixed grade programs as high as programs specifically 
aimed at high school teachers. When, the program addressed both science and math teachers, 
high school teachers generally gave higher ratings to programs which were specifically addressed 
to their grade level, as opposed to those with no discernible grade focus. The effects are not 
as strong as those for teachers of lower grade students, but are still statistically significant: 77% 
of high school teachers in single-level multi-subject projects rated the overall project quality as 
excellent, compared with 70% of high school teachers in projects with neither a specific grade 
focus nor specific subject content. 
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Teachers in upper elementary grades (5 through 8) resembled high school teachers 
in their preference for at least some focus on either a specific grade level or a specific content. 
When these teachers attended a project addressing both math and science, they uniformly gave 
higher ratings to grade-specific projects than to multi-grade projects. 

These patterns are directly related to differences in goals between elementary, middle 
grade, and high school teachers, elementary teachers place high value on goals representing 
instructional technique, and prefer TEP projects restricted to their grade level. In programs without 
a grade level restriction, elementary teachers specifically object to project staff s lack of familiarity 
with schools and teaching issues. 

High school teachers place higher value on goals relating to substantive scientific or 
mathematical content. These goals seem to have been equally satisfied in all projects which had 
either a specific science focus or a specific math focus, but not in those which either had no 
identifiable content area or tried to address all content areas. 

Patterns of participant ratings of specific results of their TEP participation resembled 
their overall quality ratings. Again, in general, ratings expressed a high degree of overall satisfaction: 
most teachers saw improvements in most of the areas covered by the survey. A principal components 
factor analysis suggested that the items in this scale fell into two broad classes, one relating to 
improved instructional skills and techniques for classroom use, the other relating to interactions 
between the participants and other teachers, project staff, or local institutions. 

In both of these groups of items, early elementary teachers who attended math programs 
focused on their grade level tended to assign more positive ratings than early elementary teachers 
attending math programs with no clear grade level focus. High school science teachers likewise 
gave higher ratings to projects with a specific grade level focus which matched their teaching 
assignment. The nature and probable explanation of these patterns is essentially similar to that 
discussed above for general satisfaction with program quality and will not be discussed here in 
detail. 

Responses to some of these items are related to the size and ethnic composition of 
project participant groups. Overall, ratings for items on the classroom techniques factor of the 
scale tended to be somewhat higher for middle-sized projects (40 to 100 participants) than for 
those which were larger or smaller. For example, 42% of teachers in middle-sized projects said 
it was very true that their TEP participation helped them "learn to update teaching practices, H 
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compared with about 32 % of those in smaller or larger projects. Similarly 61% of those in middle- 
sized projects said it was very true that TEP had "increased [their] knowledge of sources or types 
of teaching materials," compared with about 51 % of those in smaller or larger projects. 

Ratings for networking objectives, such as remaining in contact with TEP project staff 
or other participants, tended to be slightly lower in the largest projects (those with over 100 
participants) than in the small or middle-sized projects. 

Projects with greater minority teacher participation (more than 10% of participants) 
tended to receive higher satisfaction ratings on the classroom techniques factors than those whose 
participants were nearly all white and non-Hispanic. No clear pattern is evident in the relationship 
between networking objectives and ethnic mix. 

About a third of TEP participants reported that they had run into some barriers to 
applying the content or methods advocated by the TEP project they attended. Teachers in projects 
with local recruitment ~ that is, teachers whose TEP institution was based in the same community 
where they taught ~ were slightly less likely to report such barriers than teachers recruited from 
beyond the local area. In addition, teachers from projects with substantial minority representation 
(25 % or more) were slightly more likely to report such barriers than were teachers in other projects. 

Minority teachers, however, were no more likely than others to report encountering 
barriers to implementation. Teachers (of any ethnicity) in schools where a majority of children 
qualified for reduced price lunches were slightly more likely to report barriers to implementation 
than were teachers in schools with fewer than 25 % of the children so situated. This is consistent 
with the fact that almost 60% of the teachers who reported a barrier mentioned lack of funds, 
supplies, or materials as the source of the problem. 

Experienced teachers were much less likely to report barriers than those who had just 
begun teaching. Forty-two percent of teachers with five years experience or less reported at least 
one barrier, compared with 17% of those with over 35 years of experience. 

Principal Investigators 

A number of significant associations were found with respect to PI responses to their 
survey. First, the goals that a PI had for his or her TEP project were associated with the choice 
of instructional methods and materials used. As illustrations: 
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• The Pis who emphasized subject matter objectives used more formal 
teaching methods. Those Pis seeking to enhance teachers' knowledge 
of science and mathematics often used lectures; similarly, those interested 
in integrating science and mathematics knowledge often made use of 
computer-assisted instruction methods and field trips; 

• Pis who emphasized pedagogic objectives used more process-oriented 
teaching methods. Those Pis who emphasized the updating of participants' 
teaching practices, or the improvement of participants' abilities to enhance 
their students' interest in science and mathematics or their students' problem- 
solving skills, often used such teaching methods as cooperative learning, 
role play, simulations and games. 

Related to this, one factor beyond project goals was associated with the instruction method chosen: 
those Pis working with high school teachers were more likely to use lectures, while those working 
with elementary school teachers emphasized more process-intensive instructional methods. 

Second, a Pi's TEP project goals also were influenced by whether or not he or she 
developed instructional materials. For example, those Pis with pedagogic goals had higher participant 
involvement in the development of instructional materials, involved the TEP project staff in this 
process, and were more likely to use outside experts in the process as well. 

Third, a project's outcomes were associated with the instruction methods used and 
with the development of curriculum materials in the TEP project. Lecture methods, according 
to Pis, were associated with an increase in participants' subject matter knowledge. Computer-assisted 
instruction, field trips, library research, use of films and videos, and examination of instructional 
materials also were associated with the Pis' assessments of increases in participants' knowledge 
of content and teaching materials. 

On the other hand, more process-oriented methods, such as role playing, games, and 
simulations, were more likely to receive higher PI ratings of participant satisfaction with the TEP 
experience on a variety of other measures. As illustrations: more process-oriented methods (such 
as the use of cooperative learning groups) were associated with the maintenance of longer-term 
contacts between Pis and participants; the development and dissemination of new materials in 
the TEP projects was associated with the maintenance of longer-term contact with the H, with 
the use of materials in the participants' school districts, and with the subsequent presentation of 
these materials at professional conferences. Where outside experts were involved in materials 
development, the materials were more likely to be disseminated to the schools. 
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Fourth, the level of project budget, and the extent of non-NSF financial support, Mre 
associated with the choice of instructional methods. Pis with smaller budgets were more likely 
to use lecture methods, less likely to use new materials and, as a result, the participants were 
less likely to use TOP project materials in their classrooms. On the other hand, those Pis with 
larger budgets were more likely to use more process-intensive methods, to make greater use of 
other project staff, and to develop instructional materials. 

Institutional financial and other support were also associated with certain project 
characteristics. Funding support from sources other than NSF was associated with the use of 
computer-assisted instruction, instructor demonstrations, and participant experiences with hands-on 
activities. Host institution support was also associated with use of small group discussions and 
cooperative learning groups, examination of instructional materials, development of new instructional 
materials (and the involvement of participants in this process) , and participant contacts with outside 
resources. Where there was lower external support, Pis were more likely to use role play, simulations, 
and games and to make use of peer teaching. 

Fifth, support from host institutions was associated with various perceived benefits 
to participants, such as increased participant knowledge of science or mathematics content, familiarity 
with instructional materials, dissemination of project materials, subsequent participant return to 
TOP projects in staff positions, journal articles by Pis about their TOP projects, and more continuing 
collaboration between participants and the host institution. 

Sixth, a Pi's ethnicity and physical disability status were associated with project 
participation and impact characteristics. White, non-Hispanic Pis were significantly less likely 
to achieve high minority participation in their TOP projects than minority Pis. Fifty-eight percent 
of all Black participants were located in projects with Black Pis. In projects with Hispanic Pis, 
38% of all participants were also Hispanic. Similarly, Pis with physical disabilities limiting a 
major life activity were most likely to recruit participants with such disabilities; they were also 
somewhat more likely than Pis without disabilities to attract minority participants. 

Notably, minority Pis were more likely to report outcomes related to networking with 
other sources of science information and support. Pis with high Black participation were twice 
as likely as Pis with low minority participation to report both working with colleges, museums, 
etc. , and increased collaboration between scientists and mathematicians and educational specialists 
at host institutions. 
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Seventh, other PI characteristics appeared to have little to do with the selection of 
project methods and materials, or with TEP project impacts. Departmental affiliation, educational 
background (subject and level of highest earned degree), tenure status, age, ethnicity, gender, 
and experience with prior TEP awards were not related systematically to most participant outcomes. 

Finally, certain other project characteristics were associated with participation patterns. 
Pis who recruited locally were more likely to have high minority participation; general recruiting 
methods (e.g. , announcements) were less likely to attract minority participants than non-minorities. 
Conversely, where Pis made special efforts to recruit participants in minority schools, they were 
successful in attracting them. 

The distance between a participant's home and the location of the TEP project were 
associated with female participation rates; females were far less likely to participate in a project 
outside of their locality. Participants' conflicts with family responsibilities were cited as a chief 
factor by Pis for failure to achieve female recruitment goals in projects that drew participants 
from large geographic areas. 

Participant and Pis 9 Observations About Their TEP Experiences 

Principal Investigators and participants were provided with an "open-ended" opportunity 
to present various project, program, and policy ideas they had regarding the Teacher Enhancement 
Program. The following summarizes their observations. 

• Making use of "hands-on" learning techniques was a common theme - 
cited by Pis frequently as an important lesson learned, and by both Pis 
and Participants as one of the "best models" for in-service training, and 
as an important change to be made in future project, (where it had previously 
not been sufficiently utilized); 

• A greater focus on the involvement of participants in project operations 
was also considered important by Pis and participants alike - involving 
participants in the planning and setting of project goals, the use of teacher 
pairs and groups, and greater involvement of participants in the learning 
process; 

• An extremely important lesson learned, according to Pis, was the need 
for obtaining more commitment from school districts; 

• Many Pis and participants emphasized the need for more follow-up activities 
to reirtforce the in-service experience - more follow-up and support of 




participants, the formation of participant networks, and the continuing 
involvement of participants; 

• Avery important accomplishment identified by Pis and participants related 
to the participants' teaching of science and mathematics ~ upgrading 
participants' teaching practices, and improving their substantive knowledge; 

• The development of leadership capabilities was another second key theme 
among both groups - enhancing the self-confidence of participants, focusing 
on leadership development, and using exemplary teachers to teach other 
teachers; 

• Many Pis and participants took the opportunity to compliment NSF on an 
outstanding program and to comment on the high quality of NSF staff 
involved in its operation; 

• A dominant theme among Pis was the need for greater lead time between 
notification of awards and project initiation - many Pis believed that NSF 
should require earlier grant application submission in order to achieve a 
more appropriate award notification schedule; 

• The issue of more direct support to Pis by NSF staff was an important 
theme - including the need for greater availability of NSF staff to Pis, 
and better communications with Pis, including the sharing of ideas on how 
best to operate in-service programs; and 

• Many Pis felt the need for follow-up funding of successful TEP projects 
- extending the period from three to five years. 

Pis and participants provided greater detail on significant project accomplishments, 

lessons that they learned, most promising TEP models, changes they would make if running their 

projects again, and so forth. 

• The Pis most frequently mentioned, as significant impacts of their projects 
upon participants, the updating of teaching practices and improving subject 
area knowledge of participants, enhancement of participants' self-confidence, 
and formation of participant networks (see Figure 18). 

• Participants felt that the aspects of their TEP projects having the greatest 
value involved the experience with hands-on/laboratory techniques, increases 
in their knowledge and comfort with the subject matter, interaction with 
other participants, a*id new teaching techniques and curricula (see Figure 
19). 
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Figure 18 - Principal Investigators 9 Reports Upon Specific Significant Project 
Accomplishments, by Type Accomplishment 
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Figure 19 - Participants' Views of Greatest Impacts of Projects 
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The need to obtain more commitment from participants* school districts, 
and the need for a hands-on approach were the most frequently cited "lessons 
learned 11 by Pis about running the TEP projects(see Figure 20). 

Participants rated the greatest benefits of TEP project participation for 
their students as students increased their knowledge and confidence in subject 
matter, obtained greater experience with improved demonstration, 
manipulates and hands-on activities; increased their interest in science 
or math, and their exposure to new teaching methods (see Figure 21). 

According to Pis, the most promising TEP models included the use of 
role models (top teachers teaching other teachers), the use of hands-on 
methods, and the use of teacher pairs or groups to enhance learning (see 
Figure 22). 

Participants also identified various specific models, and favored hands-on 
and cooperative learning groups as models (see Figure 23). 

Were Pis to run another TEP project, lessons to be implemented included 
continuing involvement of participants, more follow-up support for partici- 
pants, more hands-on activities, more extensive recruitment activity, more 
pre-project planning, and making a greater effort to influence the school 
districts and other local organizations (see Figure 24). 

Many participants were satisfied with their TEP projects as is. Those 
suggesting changes suggested ones related to course content, solicitation 
of more inputs from participants by Pis, use of more field trips, and more 
program follow-up by Pis (see Figure 25). 

Pis would like to see NSF provide more lead time for the start-up phase 
of their projects, have more communication with Pis, provide more funding, 
and provide better access to NSF support staff (see Figure 26). 

Participants also suggested that NSF might increase funding for TEP projects, 
that the projects provide more effective outreach and recruiting, and that 
Pis have more regular follow-up with participants (see Figure 27). 
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Figure 20 - Principal Investigators 9 Reports of Specific Lessons Learned 
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Figure 21 - Participants' Reports on Benefits to Their Students 
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Figure 22 - Principal Investigators' Reports of Particular Model 
with Greatest Promise for Enhancing Teacher Skills 
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Figure 23 - Participants' Reports of a Particular Model 
with Greatest Promise for Enhancing Teacher Skills 
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Figure 24 - Principal Investigators' Reports of Important Changes 
They Would Make in Their TEP Projects 
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Figure 25 - Participants' Reports of Important Changes 
They Would Like to See Made in Their TCP Projects 
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Figure 26 - Principal Investigators' Suggestions for NSF Regarding 
Ways to Improve the Support it Provides to Pis 
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Figure 27 - Participants' Suggestions for NSF Regarding 
Ways to Improve its Operation of the TEP Program 
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Achievement of Program and Agency Objectives 

It is useful to comment upon the achievement of TEP objectives from at least four 
perspectives: those of the Teacher Enhancement Program itself, the Division of Teacher Preparation 
and Enhancement (TPE), the Directorate of Education and Human Resources (EHR), and the 
National Science Foundation as a whole. 



The TEP Programs 

A variety of specific TEP objectives were achieved: 

• The TEP projects clearly stimulated the development and documentation 
of new methods and materials for in-service education tailored to elementary, 
middle, and high-school teachers. 



• The TEP projects provided teacher participants with opportunities to learn 
discipline-specific concepts and processes and effective teaching methods, 
as reflected in the impact assessments of the Pis. 

• The Pis and participants reported as one of their major accomplishments 
the expansion of participants' knowledge of concepts and applications of 
science and mathematics. 

• Many of the Pis reported recruiting outstanding classroom teachers who 
subsequently developed leadership skills to assist in improving the teaching 
practices of their less well-prepared colleagues. 

• likewise, Pis reported recruitment of less well-prepared classroom teachers 
with demonstrated needs and expressed commitments to pursue professional 
development - an important target group for the TEP projects. 

• Many of the projects provided the opportunity for teachers to share and 
work together in local and/or regional settings in order to improve their 
own teaching under the guidance of the project staff. The amount of such 
follow-up appeared to increase over the course of the TEP program from 
1984 to 1989. 



• According to the Pis and participants, the participants established continuing 
collaborative partnerships with faculty members of schools, colleges, and 
universities, as well as personnel from a variety of other public and private 
organizations. Representatives of museums were an especially important 
source of collaborative partnerships for TEP projects. 
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• The TEP projects gave special attention to increasing student access to 
careers in science, mathematics, and technology by reaching teachers who 
are serving populations underrepiesented in the sciences, as well as teachers 
serving economically disadvantaged communities. 

• Another frequently reported impact of these TEP projects was the deepening 
of participants' knowledge and their application of improved teaching 
methods, including an emphasis upon hands-on learning techniques and 
effective applications in the laboratory. 

• One objective of the TEP project appeared to have been less often 
accomplished: participation by teachers with physical disabilities was less 
frequently reported than might have been hoped for by NSF. 



TPE Objectives 

The two major objectives of the Division of Teacher Preparation and Enhancement 
appeared, by and large, to have been accomplished in the TEP program: 

• Based upon PI and participant reports, the TEP program appeared to have 
supported well-designed projects that directly benefit the teacher-participants 
by making them more competent in the subject matter, more comfortable 
in its presentation, and more committed to their profession and their pupils ; 
and 

• In the judgment of the Pis and participants, the TEP projects have added 
to the base of know ledge about how teachers can most effectively be prepared 
and subsequently aided in enhancing their capabilities. 

Several operational objectives emphasized by TPE appeared to have been achieved 
through the TEP program as well: 

• The TEP projects appeared to have capitalized on the prestige of the NSF 
grant and exploited the Foundation's unique familiarity and relationship 
with the scientific research and education community - perhaps best 
illustrated by the effective leveraging of NSF funds; 

• As was demonstrated by the broad base of project staff support (both paid 
and voluntary), the projects stimulated collaboration among such 
organizations as the Federal government, colleges and universities, state 
and local education agencies, business and industry, cultural institutions, 
and media, and among such partners as scientists and science educators, 
other teachers, school administrators, and parents; 
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• According to PI reports, the TOP projects have led to some self-sustaining 
networks among these varied elements - cooperative patterns that will 
continue to function, supported by non-NSF fiinds; and 

• The individual TOP projects did a good job of focusing resources and using 
in-service approaches that appeared to be widely applicable to strengthening 
science and mathematics education. 

Many TOP projects involved the formulation, testing, and refining of various approaches 
to in-service education - with three out of four Pis reporting that they utilized evaluation methods 
to assess project accomplishments objectively. 

EHR Objectives 

The Directorate for Education and Human Resources defines and funds programs and 
projects that support NSF's educational mission. This directorate has four long-range goals to 
help ensure that: 

• high quality primary and secondary education in science and mathematics 
is available to every child in the United States; 

• those who select careers in science, mathematics, and engineering have 
available the best possible professional education in their disciplines; 

• opportunities are available at the college level for interested non-specialists 
to broaden their science backgrounds; and 

• sufficient support is available for science education outside the classroom 
to encourage and maintain public interest in, and awareness of, scientific 
and technological developments. 

The roughly 600 TOP projects included in this evaluation reached an estimated 63,000 
teachers throughout the United States, at all school levels, and in a variety of communities. The 
TOP projects also have addressed the major subject areas of science and mathematics instruction. 
The TOP projects appear to have contributed to the long-range goals of EHR. 

NSF Objectives 

Finally, the National Science Foundation seeks to leverage application of its resources 
and is strongly committed to the principle of cost sharing in its projects. Cost sharing among 
collaborating partners is strongly encouraged for all grant recipients - especially where the goals 
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include the continuation and maintenance of project activities by local institutions or self-sustaining 
networks after NSF funding has terminated. By achieving a 53% leverage (including both financial 
and in-kind contributions), the TOP projects have done especially well in this regard. 

The Foundation is also especially concerned about the underrepresentation of women, 
minorities, and the physically disabled in careers in mathematics, engineering, and the sciences. 
Projects involving members of these groups as principal investigators or staff, and as participants, 
are especially encouraged. A large percentage of both Pis (22.6%) and participants (57.4%) are 
female. 

TOP projects have attracted some Hs who have physical disabilities, with five percent 
reporting such disabilities. The percentage of TOP participants with disabilities is lower, however, 
with around three percent of participants having some form of physical disability. 

The picture with respect to minority participation among TOP Pis and participants 
is less encouraging. The incidence of Pis who are members of minority groups was low (5%). 
Participation by minorities, according to self-reported minority status, was low, with 7.9% of 
participants being black, 1.7% Hispanic, and 1.0% native Americans or Pacific Islanders. Minority 
TOP Pis did better in attracting minority applicants than did non-minority Pis. 



Implications for Changes in TEP Policies and Programs 

A number of program and policy implications arise from this study. 

• TOE should emphasize that prospective grant applicants need to obtain 
commitments of non-NSF financial or in-kind support for their TOP projects. 
Aside from consistency with NSF policies, such support has demonstrable 
benefits associated with it, including greater diversity of teaching methods 
(including computer-assisted instruction), development of curriculum 
materials, dissemination of curriculum materials, higher achievement of 
project goals, greater continuing use of other community scientific resources, 
longer term contact between Pis and participants, publication of journal 
articles, and continuation of support after completion of the TOP grant. 

• TOE should carefully examine the program funding cycle to determine 
if it is possible to initiate requests for proposals under a schedule that would 
result in an earlier award date - thus expanding the amount of recruitment 
time available to successful awardees. 



50 58 



TPE may wish to examine the caseload of its program staff to insure that 
sufficient time is available for the provision of program support to TEP 
awardees. 

With respect to the funding cycle, three years appears will be appropriate. 
It may be useful to prepare a set of criteria for present awardees, however, 
indicating specifically the conditions that must be met if a project is to 
receive an optional, two year follow-on. In this case, there are, of course, 
trade-offs between maintaining a truly superior project serving a significant 
number of participants, versus funding a new PI who has good ideas but 
is untried. 

The finding that TEP projects that concentrated their focus on more delimited 
subject matter and grade span (especially for kindergarten and early grade 
mathematics) were more highly rated by participants than some of the less 
delimited projects deserves some examination and consideration with regard 
to grant application guidelines and in the screening of grant applications. 

It may be useful for TEP to determine the match between the location of 
the Pis (and the number and type of participants that they serve) and the 
distribution of school districts and various population groups. It would 
be especially important for NSF to know the extent to which there are 
significant mismatches between the location of school systems needing 
services and the location of Pis providing the services. One finding of 
this study appears especially important in this respect; female participants 
are less likely to be attracted to projects located far from their districts. 

TPE staff should consider steps that would increase the participation of 
Pis from underrepresented groups. While the population of TEP awardees 
would appear to include a reasonable percentage of individuals with physical 
disabilities, only one in five of TEP awardees is a woman, and (among 
awards where ethnicity or race is known), only one in ten are members 
of under-represented groups. (As a corollary, it is clear from the study 
results that Pis who are members of underrepresented groups are notably 
effective in recruiting participants from these same groups.) 

Turning to underrepresented groups of participants found in the TEP projects 
(as distinct from the Pis themselves), TPE staff should also make every 
effort to alert TEP Pis to the importance of actively recruiting members 
of underrepresented groups in science and mathematics, particularly in 
any local school with a higher proportion of such persons among its faculty. 
It is clear that where Pis do target their recruiting to districts with under- 
represented groups, their level of participation in the projects increases. 
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